Can Design Choices for Language-Based Editors be Analysed with Keystroke-Level Models?

We have been concerned for some time with the lack of rigorous experimental evaluation of design options chosen for tools used by software engineers. In a series of studies using various evaluation techniques we built Keystroke-Level Models and conducted an empirical usability study of a design issue (choice of editing paradigm for language-based editors) that has reached a `subjective stalemate' in the research community. The KLM analysis enabled us to predict usage diierences and while this was useful we also noticed several problems, in particular we were concerned about the estimated value and placement of the memory operator. By utilising the same tasks in the usability study as in the KLM analysis, we were able to compare results from both evaluations and eeectively validate the overall KLM estimates and the speciic operator values involved.